add periodic job definitions for ephemeral volumes perf/scalability test cases #13471

mucahitkurt · 2019-07-16T20:49:27Z

Periodic job definitions for ephemeral volumes performance/scalability test cases

Meaningful when the test cases PR is merged.

I need support for TODO items.

k8s-ci-robot · 2019-07-16T20:49:28Z

Welcome @mucahitkurt!

It looks like this is your first PR to kubernetes/test-infra 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/test-infra has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2019-07-16T20:49:35Z

Hi @mucahitkurt. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

msau42 · 2019-07-16T22:06:26Z

/ok-to-test

msau42 · 2019-07-16T22:09:37Z

/assign @wojtek-t

wojtek-t

Couple comments, but seems reasonable to me.

wojtek-t · 2019-07-17T11:11:01Z

config/jobs/kubernetes/sig-scalability/sig-scalability-periodic-jobs.yaml

@@ -392,3 +392,297 @@ periodics:
  annotations:
    testgrid-dashboards: sig-scalability-experiments
    testgrid-tab-name: gce-private-cluster-correctness
+# max volumes per pod test cases for ephemeral volumes


Given that for now those are a bit experimental tests, I would prefer splitting them into a separate file.
Sth like sig-scalability-experimental-periodic-jobs.yaml

This would make it clear to users that if they won't work, it may be expected, etc.
[if we will do that, we should also move the private-clustester tests there]

@mm4tt - thoughts? concerns?

ok, I'm waiting for your final decision about that.

I totally agree with Wojtek, +1 to his point.

I moved the new storage test jobs and private-cluster job to a new file sig-scalability-experimental-periodic-jobs.yaml.

wojtek-t · 2019-07-17T11:12:21Z

config/jobs/kubernetes/sig-scalability/sig-scalability-periodic-jobs.yaml

+- name: ci-kubernetes-storage-scalability-max-emptydir-vol-per-pod
+  interval: 1h
+  labels:
+  #TODO which labels I need?


Seems fine - remove a TODO.

wojtek-t · 2019-07-17T11:20:23Z

config/jobs/kubernetes/sig-scalability/sig-scalability-periodic-jobs.yaml

+      - --test-cmd-args=--nodes=1
+      - --test-cmd-args=--provider=gce
+      - --test-cmd-args=--report-dir=/workspace/_artifacts
+      - --test-cmd-args=--testconfig=testing/experimental/storage/pod-startup/ephemeral-volumes/1_node/config.yaml


nit: I think it would be a bit more intuitive to add one more override for emptydir (even though it's the default) so that we will know what exactly is happening here (and for consistency with other definitions).

I agree, explicit is better. I also will do the same thing for default test case(that's max volume per pod), this will be another PR for perf-tests.

kubernetes/perf-tests#653

wojtek-t · 2019-07-17T11:21:19Z

config/jobs/kubernetes/sig-scalability/sig-scalability-periodic-jobs.yaml

+      - --test-cmd-args=--nodes=1
+      - --test-cmd-args=--provider=gce
+      - --test-cmd-args=--report-dir=/workspace/_artifacts
+      - --test-cmd-args=--testconfig=testing/experimental/storage/pod-startup/ephemeral-volumes/1_node/config.yaml


same here about override for emptydir

wojtek-t · 2019-07-17T11:22:45Z

@mm4tt

wojtek-t · 2019-07-18T18:12:46Z

/retest

wojtek-t · 2019-07-18T18:26:06Z

test-infra-bazel seems to be failing - @mucahitkurt - could you please take a look if that's a real issue?

mucahitkurt · 2019-07-18T18:56:59Z

test-infra-bazel seems to be failing - @mucahitkurt - could you please take a look if that's a real issue?

fixed, I forgot the periodics: for the new file, thanks!

wojtek-t · 2019-07-19T06:35:20Z

This looks fine, but unfortunately there is a conflict now.
Can you please rebase?

[Also feel free to ask someone else for lgtm]

/approve

k8s-ci-robot · 2019-07-19T06:35:28Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mucahitkurt, wojtek-t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~config/jobs/kubernetes/sig-scalability/OWNERS~~ [wojtek-t]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

wojtek-t · 2019-07-19T07:04:22Z

config/jobs/kubernetes/sig-scalability/sig-scalability-experimental-periodic-jobs.yaml

+      - --test_args=--ginkgo.flakeAttempts=2 --ginkgo.skip=\[Serial\]|\[Disruptive\]|\[Flaky\]|\[Feature:.+\]|\[DisabledForLargeClusters\] --minStartupPods=8 --node-schedulable-timeout=90m
+      - --timeout=240m
+      - --use-logexporter
+  annotations:


Actually, I just sent out a PR to unify stuff a bit in our configs: #13516
So if you could move annotations right after labels and add an empty line between configs, it would be great.

fixed, thanks!

mucahitkurt · 2019-07-19T15:12:42Z

This looks fine, but unfortunately there is a conflict now.
Can you please rebase?

[Also feel free to ask someone else for lgtm]

/approve

Rebased, thanks!

…ility test cases Signed-off-by: Mucahit Kurt <[email protected]>

wojtek-t · 2019-07-22T05:53:37Z

/lgtm

@mucahitkurt - thanks a lot!
If you could take a look some time today (during your dat) if the results there are looking reasonable it would be great (I'm assuming there may be some issues with those tests, but the experiments are basically to confirm/reject those hypothesis)

k8s-ci-robot · 2019-07-22T05:53:39Z

LGTM label has been added.

Git tree hash: 8b025d57d88d8ecd17e32013420d337df1381652

k8s-ci-robot · 2019-07-22T06:02:55Z

@mucahitkurt: Updated the job-config configmap in namespace default using the following files:

key sig-scalability-experimental-periodic-jobs.yaml using file config/jobs/kubernetes/sig-scalability/sig-scalability-experimental-periodic-jobs.yaml
key sig-scalability-periodic-jobs.yaml using file config/jobs/kubernetes/sig-scalability/sig-scalability-periodic-jobs.yaml

In response to this:

Periodic job definitions for ephemeral volumes performance/scalability test cases

Meaningful when the test cases PR is merged.

I need support for TODO items.

Fix #13470

cc @wojtek-t @msau42

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mucahitkurt · 2019-07-22T15:43:03Z

/lgtm

@mucahitkurt - thanks a lot!
If you could take a look some time today (during your dat) if the results there are looking reasonable it would be great (I'm assuming there may be some issues with those tests, but the experiments are basically to confirm/reject those hypothesis)

@wojtek-t When I check the tests from testgrid, all test cases are passing, but for some test cases, I can't see the podstartup latency artifacts inside the artifacts directory, like this one, I can see for this one.

We set the pod startup latency threshold to 5second, when I check the build logs of these test cases I see pod startup latencies over 5seconds, can be the reason why pod startup artifacts are not generated?

Is there any dashboard like environment to see the how pod startup latencies are changed across each test run? I remember you mentioned something like that in my PR, but I can't access to it right now(http 500, I think github has some issues).

I think I need some support to read/evaluate these test results.

wojtek-t · 2019-07-23T09:25:00Z

@mucahitkurt - kubernetes/perf-tests#667 opened

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jul 16, 2019

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jul 16, 2019

k8s-ci-robot requested review from mborsz and wojtek-t July 16, 2019 20:49

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. area/config Issues or PRs related to code in /config sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Jul 16, 2019

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 16, 2019

k8s-ci-robot assigned wojtek-t Jul 16, 2019

wojtek-t reviewed Jul 17, 2019

View reviewed changes

mucahitkurt force-pushed the scalability/storage/add-periodic-jobs-for-ephemeral-volumes-perf-tests branch from 07649f4 to cd50373 Compare July 17, 2019 19:25

mucahitkurt mentioned this pull request Jul 17, 2019

Update override files for ephemeral volumes perf test cases kubernetes/perf-tests#653

Merged

mucahitkurt force-pushed the scalability/storage/add-periodic-jobs-for-ephemeral-volumes-perf-tests branch 2 times, most recently from be8ef18 to 8ba3d05 Compare July 18, 2019 17:34

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 18, 2019

mucahitkurt force-pushed the scalability/storage/add-periodic-jobs-for-ephemeral-volumes-perf-tests branch from 8ba3d05 to 2084f3f Compare July 18, 2019 17:41

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 18, 2019

mucahitkurt force-pushed the scalability/storage/add-periodic-jobs-for-ephemeral-volumes-perf-tests branch from 2084f3f to 7c41598 Compare July 18, 2019 18:32

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 19, 2019

wojtek-t reviewed Jul 19, 2019

View reviewed changes

mucahitkurt force-pushed the scalability/storage/add-periodic-jobs-for-ephemeral-volumes-perf-tests branch 2 times, most recently from 86bddfc to bd83f91 Compare July 19, 2019 15:09

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 19, 2019

add periodic job definitions for ephemeral volumes performance/scalab…

556339c

…ility test cases Signed-off-by: Mucahit Kurt <[email protected]>

mucahitkurt force-pushed the scalability/storage/add-periodic-jobs-for-ephemeral-volumes-perf-tests branch from bd83f91 to 556339c Compare July 20, 2019 02:33

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 20, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 22, 2019

k8s-ci-robot merged commit 0012145 into kubernetes:master Jul 22, 2019

mucahitkurt deleted the scalability/storage/add-periodic-jobs-for-ephemeral-volumes-perf-tests branch July 22, 2019 20:21

mucahitkurt mentioned this pull request Sep 7, 2019

REQUEST: New membership for mucahitkurt kubernetes/org#1166

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add periodic job definitions for ephemeral volumes perf/scalability test cases #13471

add periodic job definitions for ephemeral volumes perf/scalability test cases #13471

mucahitkurt commented Jul 16, 2019

k8s-ci-robot commented Jul 16, 2019

k8s-ci-robot commented Jul 16, 2019

msau42 commented Jul 16, 2019

msau42 commented Jul 16, 2019

wojtek-t left a comment

wojtek-t Jul 17, 2019

mucahitkurt Jul 17, 2019

mm4tt Jul 18, 2019

mucahitkurt Jul 18, 2019

wojtek-t Jul 17, 2019

wojtek-t Jul 17, 2019

mucahitkurt Jul 17, 2019

mucahitkurt Jul 17, 2019

wojtek-t Jul 17, 2019

wojtek-t commented Jul 17, 2019

wojtek-t commented Jul 18, 2019

wojtek-t commented Jul 18, 2019

mucahitkurt commented Jul 18, 2019

wojtek-t commented Jul 19, 2019

k8s-ci-robot commented Jul 19, 2019

wojtek-t Jul 19, 2019

mucahitkurt Jul 19, 2019

mucahitkurt commented Jul 19, 2019

wojtek-t commented Jul 22, 2019

k8s-ci-robot commented Jul 22, 2019

k8s-ci-robot commented Jul 22, 2019

mucahitkurt commented Jul 22, 2019

wojtek-t commented Jul 23, 2019

add periodic job definitions for ephemeral volumes perf/scalability test cases #13471

add periodic job definitions for ephemeral volumes perf/scalability test cases #13471

Conversation

mucahitkurt commented Jul 16, 2019

k8s-ci-robot commented Jul 16, 2019

k8s-ci-robot commented Jul 16, 2019

msau42 commented Jul 16, 2019

msau42 commented Jul 16, 2019

wojtek-t left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wojtek-t commented Jul 17, 2019

wojtek-t commented Jul 18, 2019

wojtek-t commented Jul 18, 2019

mucahitkurt commented Jul 18, 2019

wojtek-t commented Jul 19, 2019

k8s-ci-robot commented Jul 19, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mucahitkurt commented Jul 19, 2019

wojtek-t commented Jul 22, 2019

k8s-ci-robot commented Jul 22, 2019

k8s-ci-robot commented Jul 22, 2019

mucahitkurt commented Jul 22, 2019

wojtek-t commented Jul 23, 2019